gc(policy): the ArenaBytes major trigger and its consumers measure old space, not the copying nursery's reserved capacity - #9951
Draft
proggeramlug wants to merge 4 commits into
Conversation
added 2 commits
September 7, 2026 07:30
Track reserved nursery capacity alongside the arena total and use the non-nursery remainder for ArenaBytes arming, rebaselining, debt, tiny-parse pressure, memory-pressure clamps, and idle right-sizing. Keep promotion transfers visible as old-space growth. Extend GC diagnostics with the compared old-space total and excluded nursery capacity, and add sabotage-oriented coverage for nursery-only pressure, promotion-driven old-space pressure, and the idle right-size handoff.
Record the implementation SHA, source map, sabotage points, disk-gated test status, performance predictions, and exact perrymaster gate and measurement request for the old-space ArenaBytes change.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Account for the mandatory replacement Eden head after whole-block promotion, and assert that only promoted capacity enters old-space pacing.
Record the block ownership audit, base-difference result, implementation SHA, and local validation gates for the one-block test correction.
This was referenced Sep 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runtime-only, on main
616a2cb84. Written by codex from the campaign's major-cycle and allocation accounting on cc; not yet compiled (the authoring host had no disk) — perrymaster's gate ladder and rows will be appended here. GC-adjacent: needsrun-extended-tests.Why
On cc (4-turn 3300-char replies,
PERRY_GC_DIAG), every budgeted major cycle inside a turn is armed byArenaBytes(arena_total ≥ next_base) while old-space reclaimable sits at its baseline — nothing new to reclaim in old space.arena_totalcounts the copying nursery's reserved from-space capacity, so a nursery filling toward its cap crosses the base by itself. The cycle's cost is not the old-gen mark (5–7 ms for 50–65 MB of old space) but sweeping 46–64 MB of dead Eden bytes in place (69 % of the cycle, ~85 ms each): the copying minor's work done the expensive way, ≈ 50 ms of mutator charge per turn on the default nursery and more on larger ones. Everything on cc allocates through the nursery; old space grows only by promotion (0.1–1.5 MB per turn). The same nursery-inclusive quantity made #9838's tiny-parse guard read a large nursery as pressure (up to 159 non-collecting forced-collection requests in one turn), and the adaptive tenuring arm's turn-1 excursion (TN4) is this cycle armed by the arena crossing its S=2 startup minors shift.What changes (a unit change, not a threshold change)
ARENA_TOTAL_BYTESgainsNURSERY_RESERVED_BYTES, the reserved capacity owned by Eden and both survivor semispaces, maintained where blocks are created, reset, released and quarantined; in-place promotion transfers capacity from the nursery counter to old so promotion is visible as old-space growth.old_space_total_bytes()=arena_total − nursery reserved. TheArenaBytespredicate compares it with the armed base; the rebaseline (next_base; the 16 MiB headroom floor and the adaptive step, ceiling and promotion-runway credit are untouched), the debt scaling, the tiny-parse pressure test and the explicit memory-pressure clamp use the same quantity; idle right-sizing evaluates old-space live bytes against old-space capacity with its existing rules. Nursery occupancy stays owned by the young-scavenge cap and is handed to a copying minor. The safepoint deferral slack valve remains whole-arena by design (it bounds RSS while a collection waits and does not arm).[gc-trigger]printsarena_total=, theold_total=actually compared andnursery_excluded=;[gc-budgeted] startprintsarming_reason=with the same totals;[gc-arena-rebaseline]prints the old-space total and the excluded nursery capacity on one line.Tests (named; sabotage stated in the campaign report; not yet executed)
arena_trigger_old_space: a nursery filled with dead objects to the real cap crosses whole-arena bytes but not old bytes, arms no budgeted cycle, and the next copying minor reclaims it (restoringarena_total_bytes()in the comparison fails it); promotion past the unchanged headroom with an empty nursery arms exactly one cycle (removing the nursery-counter transfer in promotion fails it); the bounded idle-window right-size test kept with its sabotage named. Existing trigger, rebaseline, debt-pacer, copying-survival, host-safepoint, arena-right-size and idle-reclaim coverage updated only where its unit is now old-space capacity.Predictions (falsifiers on the cc 4-turn diag run vs main)
Budgeted cycles armed by
ArenaByteswith old-space reclaimable at baseline: 3 → 0 per run; in-turn budgeted mutator charge 169–213 ms → ≈ 0; minors per turn and old-space occupancy unchanged; idle-window reclaim unchanged; non-collecting tiny-parse requests → ≈ 0; paired turn CPU ≈ −2 % at 3300 and more at 400; peak and settled RSS unchanged or lower. Two consequences to measure on the same runtime: a 64 MB nursery no longer arms majors (NS2's +13 % peak), and the adaptive tenuring arm's turn-1 major should disappear without a lock change.Gate history
008fa075dpicked onto main504e180d0on perrymaster: default build green, archive feature set green,nm2951 exports present; runtime suite 3231 passed / 1 failed / 4 ignored — 7 of the 8 named tests green (botharena_right_sizefamilies, the idle right-size test, andfull_dead_nursery_does_not_arm_budgeted_cycle_and_next_minor_reclaims_it, i.e. the arming side works). The failure is this PR's ownpromotion_past_old_space_headroom_arms_one_budgeted_cycle_with_empty_nurseryatarena_trigger_old_space.rs:145: "promotion transfers capacity without reserving more" left 19 922 944 right 18 874 368 — exactly oneBLOCK_SIZE— reproducible alone. Either the nursery-reserved counter misses one block-creation arm, or the in-place promotion reserves a fresh old head the expectation must state. Follow-up in progress; the same test is being run on the branch's own base to separate a bookkeeping miss from a base difference. No rows yet.0fa6101bb(this head's code): the one-block delta was by construction —reset_young_after_promotionreserves a mandatory fresh 1 MiB Eden head after transferring the filled nursery blocks to old space — so the test now expects exactly one new block and asserts more precisely: the nursery's capacity after promotion equals exactly the replacement Eden head, and old-space pacing gains exactly the promoted capacity. The same failure reproduced on the branch's own base, and no relevant commit exists between that base and the measurement base. Local gates on this head: the exact test 1 passed; runtime suite 3261 passed, 0 failed, 4 ignored; archive feature-set build green; default compiler build green; rustfmt, diff and file-size checks green. Perrymaster runs its ladder and the rows next.Measured — AT2 (perrymaster, main
504e180d0+ this PR's two commits, runtime relinked on main's cache; gate green: suite 3232/0, all 8 named tests, 92 family tests,nm)The change does what it says: on a 4-turn 3300-char run the budgeted cycles go 3 → 0 (main: startup ArenaBytes full 292 ms + turn-1 94 ms + turn-3 95 ms of step), tiny-parse forced collections 12 → 0, exit
step_us1765 → 1443 ms; at 400 chars 2 → 0.It is not a CPU win and it costs RSS in the multi-turn regime at the default tenuring: 1-turn paired rows 3300 −0.4 / +3.7 / +1.5 / +1.1 / +1.5 % (median +1.5 %), 400 flat, peak equal; 4-turn: RSS after turns 1–4 main 643 / 616 / 621 / 620 MB vs this PR 627 / 861 / 859 / 854, peak 653 → 967 MB (+48 %), turn-2 CPU +0.55 s. Mechanism: with the adaptive policy collapsing to S=1 at the first minor (occupancy rule), every minor promotes eden blocks in place into old space and takes fresh nursery blocks; the ArenaBytes-armed majors that this PR removes were the only thing sweeping and returning that capacity between turns, so old occupancy stays at 51–55 MB while the arena grows a block-set per minor. With S=2 pinned on the same runtime the picture inverts: 0 budgeted cycles, peak 611–616, RSS 589–596 flat across four turns, promoted 12 MB in turns 2–4 — the best multi-turn profile measured on any arm.
Correctness finding: with the adaptive policy (S 4 → 1 at the first minor) cc hit a TypeError in turn 3 in 2 of 2 rounds (
Cannot read properties of undefined (reading 'inst')/(reading 'memoizedState'), same call chain, two different undefined reads: a live object lost), no panic, no runtime stderr; 2 of 3 such runs on this runtime, 0 of 2 with S=2 pinned, 0 of 1 on main. A discriminator (main with S=1 pinned; this runtime underPERRY_GC_VERIFY_EVACUATION/PERRY_GC_PROTECT_FROMSPACE) is queued. The promotion-reservation change in the second commit combined with promote-on-first-copy is the first suspect.Reading: not landing as built. The trigger's unit is right, but it removed a job the majors were doing; the coherent design is this change together with a policy that never sits at S=1 on this workload (#9949) and a copying minor that right-sizes the arena after a flip instead of relying on a major — after the crash is explained.
PERRY_GC_VERIFY_EVACUATION=1panics on this tree (gc/malloc.rs:527: RefCell already borrowed) — being fixed separately so the verifier can be used on GC branches.